Flower Image Classifier¶
Project¶
Machine learning is getting incorporated more and more into everyday apps. Soon, anything with a camera will use image recognition which is based on deep learning models. The goal of the project is to build such a model for the purpose of recognizing flower species and use it in a simple application. The app will recognize flower types that are on the images.
Import resources¶
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import json
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorflow_datasets as tfds
import tensorflow_hub as hub
tfds.disable_progress_bar()
import logging
logger = tf.get_logger()
logger.setLevel(logging.ERROR)
# The new version of dataset is only available in the tfds-nightly package
%pip --no-cache-dir install tfds-nightly --user
# need to restart the kernel after it is run for the first time
%%capture
# Download data to default local directory "~/tensorflow_datasets"
!python -m tensorflow_datasets.scripts.download_and_prepare --register_checksums=True --datasets=oxford_flowers102
# Load the dataset with TensorFlow Datasets
dataset, dataset_info = tfds.load('oxford_flowers102', as_supervised = True, with_info = True)
# Create a training set, a validation set and a test set
training_set = dataset['train']
validation_set = dataset['validation']
test_set = dataset['test']
Explore¶
# Get the number of examples in each set from the dataset info
print("Number of examples:")
num_training_examples = dataset_info.splits['train'].num_examples
print("- training", num_training_examples)
num_validation_examples = dataset_info.splits['validation'].num_examples
print("- validation", num_validation_examples)
num_test_examples = dataset_info.splits['test'].num_examples
print("- test", num_test_examples)
# Get the number of classes in the dataset from the dataset info
num_classes = dataset_info.features['label'].num_classes
print("\nNumber of classes", num_classes)
# Print the shape and corresponding label of 3 images in the training set
print("Shape and label of the first 3 images:")
for image, label in training_set.take(3):
print('\u2022 shape', image.shape, '\n\u0020 label', label.numpy())
# Plot 1 image from the training set and its label
for image, label in training_set.take(1):
plot_title = label.numpy()
plt.imshow(image, cmap = plt.cm.binary)
plt.title(plot_title)
plt.colorbar()
plt.show()
Label mapping¶
Load the flower names corresponding to the labels.
with open('label_map.json', 'r') as f:
class_names = json.load(f)
# Plot the image again, this time with corresponding class name
for image, label in training_set.take(1):
plot_title = class_names[str(label.numpy())]
plt.imshow(image, cmap = plt.cm.binary)
plt.title(plot_title)
plt.colorbar()
plt.show()
Create pipeline¶
BATCH_SIZE = 64
IMG_SHAPE = 224
# Normalize color range of an image from 0-255 to 0-1
def normalize(image, label):
image = tf.cast(image, tf.float32)
image /= 255
return image, label
# Resize image
def resize(image, label):
image = tf.image.resize(image, (IMG_SHAPE, IMG_SHAPE))
return image, label
# Apply the transformations and return a pipeline
def batchesOfImages(set):
return set.cache() \
.shuffle(num_training_examples // 4) \
.map(resize) \
.batch(BATCH_SIZE) \
.map(normalize) \
.prefetch(1)
# Create a pipeline for each set
training_batches = batchesOfImages(training_set)
validation_batches = batchesOfImages(validation_set)
test_batches = batchesOfImages(test_set)
Build and train the classifier¶
The data is ready. Now is the time to build the model. Image classification is a challenging problem, for this reason the model will be built with the use of a pre-trained neural network. This approach is called transfer learning and it involves tuning the existing network that has been trained for a similar task.
This network is MobileNet and it comes from TensorFlow Hub repository. It has been trained on ImageNet, a massive dataset with over 1 million labeled images in 1000 categories. There is a lot of intelligence stored in this model which will be helpful in building the image classifier for flowers.
Because MobileNet is a network trained for a general image recognition, the best approach would be to tune it by just replacing the output layer and training it for classifying flowers. The fact that flowers dataset is small is another argument for keeping the weights of the pre-trained network constant.
Load pre-trained network¶
It will be responsible for extracting the features of images.
# Load MobileNet
URL = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4"
feature_extractor = hub.KerasLayer(URL, input_shape=(IMG_SHAPE, IMG_SHAPE, 3))
# Set its weights constant
feature_extractor.trainable = False
Build the model¶
Add output layer with 102 output nodes.
model = tf.keras.Sequential([
feature_extractor,
tf.keras.layers.Dense(102, activation = 'softmax')
])
model.summary()
Train the model¶
# Train the model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
EPOCHS = 10
history = model.fit(training_batches,
epochs = EPOCHS,
validation_data = validation_batches)
Plot loss and accuracy¶
epochs = range(1, 11)
plt.subplot(1, 2, 1)
plt.plot(epochs, history.history['loss'], 'b--', label = 'training')
plt.plot(epochs, history.history['val_loss'], 'b', label = 'validation')
plt.legend()
plt.title("Loss in trainig and validation")
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.subplot(1, 2, 2)
plt.plot(epochs, history.history['accuracy'], 'g--', label = 'training')
plt.plot(epochs, history.history['val_accuracy'], 'g', label = 'validation')
plt.legend()
plt.title("Accuracy in training and validation")
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.tight_layout() # to make plots not overlap
plt.show()
Training result is pretty good at about 80% validation accuracy. Although the distance between the curves grows significantly which indicates overfitting.
Test the model¶
Real test on a previously unseen data, the testing set. It will give a good estimate of a model's performance in practice.
model.evaluate(test_batches)
Test results are not much different from the validation results. The model performs at about 77% accuracy.
Save the model¶
# Save the model as a Keras model
save_filename = "image_classifier"
save_filepath = './{}.h5'.format(save_filename)
model.save(save_filepath)
Load the model¶
model = tf.keras.models.load_model(
save_filepath,
custom_objects = {'KerasLayer' : hub.KerasLayer} # model uses KerasLayer which is a part of hub.KerasLayer
)
model.summary()
Inference for classification¶
Now the trained network is going to be used for prediction. With this in mind, predict function will be created. It will take an image, a model, k value and will return top k most likely classes with corresponding probabilities.
Image pre-processing¶
Before being processed by the model, an image needs to be resized to 224x224 and its colors range needs to be normalized.
# Create the process_image function
# Resize image to 224x224 and normalize the color range
def process_image(image):
image = tf.cast(image, tf.float32)
image = tf.image.resize(image, (IMG_SHAPE, IMG_SHAPE))
image /= 255
return image
Use a test image to see how it works.
from PIL import Image
image_path = './test-images/hard-leaved_pocket_orchid.jpg'
im = Image.open(image_path)
test_image = np.asarray(im)
processed_test_image = process_image(test_image)
fig, (ax1, ax2) = plt.subplots(figsize=(10,10), ncols=2)
ax1.imshow(test_image)
ax1.set_title('Original Image')
ax2.imshow(processed_test_image)
ax2.set_title('Processed Image')
plt.tight_layout()
plt.show()
Inference¶
# Create the predict function
def predict(image_path, model, top_k):
# Prepare image
image = Image.open(image_path)
image = np.asarray(image)
image = process_image(image)
# Add extra dimension expected by the model
image = np.expand_dims(image, axis = 0) # Change shape from (224, 224, 3) to (1, 224, 224, 3)
# Predict
predictions = model.predict(image)[0]
# Get indexes of top k predictions
sorted_indexes = np.argsort(predictions)[::-1] # Sort indexes, in reverse (descending) order
top_k_indexes = sorted_indexes[0:top_k]
# Extract probabilites and classes based on indexes
probabilities = [predictions[i] for i in top_k_indexes]
classes = [class_names[str(i + 1)] for i in top_k_indexes] # +1 because class_names key range is from 1 to 102
return probabilities, classes
Prediction¶
Let's use the model and see how it performs. There are 4 sample images to classify:
- cautleya_spicata.jpg
- hard-leaved_pocket_orchid.jpg
- orange_dahlia.jpg
- wild_pansy.jpg
Plot the model's predictions against each image and its true class.
# Function plotting image and its true class against model's predictions
def display_prediction(image_path, true_class, model, top_k):
image = Image.open(image_path)
image = np.asarray(image)
probabilities, classes = predict(image_path, model, top_k)
fig, (ax1, ax2) = plt.subplots(figsize = (10, 10), ncols = 2)
ax1.imshow(image)
ax1.set_title(true_class)
ax1.axis('off')
y_ticks = np.arange(top_k)
ax2.barh(y_ticks, probabilities)
ax2.set_aspect(0.1)
ax2.set_yticks(y_ticks)
ax2.set_yticklabels(classes)
ax2.set_title('Class Probability')
ax2.set_xlim(0, 1.1)
ax2.invert_yaxis() # labels read top-to-bottom
plt.tight_layout()
# Test images data
test_image_paths = [
'./test-images/cautleya_spicata.jpg',
'./test-images/hard-leaved_pocket_orchid.jpg',
'./test-images/orange_dahlia.jpg',
'./test-images/wild_pansy.jpg'
]
test_image_true_classes = [
'cautleya spicata',
'hard-leaved pocket orchid',
'orange dahlia',
'wild pansy'
]
# Plot results for each image
for (image_path, true_class) in zip(test_image_paths, test_image_true_classes):
display_prediction(image_path, true_class, model, 5)
The results for sample images are promising. The model was correct each time. It was certain about two images (hard-leaved pocket orchid and wild pansy), quite confident about one image (cautleya spicata), but it barely recognized the flower type on the fourth image (orange dahlia).
Generate report¶
!!jupyter nbconvert *.ipynb